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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the 
fee set forth in 37 CFR 1 .17(e), was filed in this application after final rejection. 
Since this application is eligible for continued examination under 37 CFR 1.114, 
and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the 
previous Office action has been withdrawn pursuant to 37 CFR 1 .1 14. 
Applicant's submission filed on July 12, 2010 has been entered. 

Response to Amendment 

2. The Examiner acknowledges the amended claims filed on July 12, 2010. 
Claims 1, 40 and 44 have been amended. Claims 6, 8-10, 21-39, 41, 43, 45 
and 48-60 have been cancelled. 

Response to Arguments 

3. Applicant's arguments with respect to claims 1 and 44 have been 
considered but are moot in view of the new grounds of rejection. 

4. Applicant's arguments filed July 12, 2010 have been fully considered but 
they are not persuasive. The Applicant argues the following: 

a. The Examiner has correctly stated that Metcalfe in view of 
Takahashi and Matsumoto suggests that "by adjusting the compression of 
the video being captured, it is possible to record the monitoring image 
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data having a high degree of importance as much as possible" (final office 
action, p. 36). More specifically, Metcalfe in view of Takahashi and 
Matsumoto suggests that the stored data can be "recompressed . . . 
depending on a degree of importance and increasing the residual 
capacity" (Matsumoto, col. 7, 11. 31-35). As such, "it is possible to ensure 
the picture quality of the important monitoring image data while 
maintaining a picture recording time for a predetermined monitored time, 
and furthermore, to reliably record the monitoring image data when an 
alarm is generated" (id., col. 7, 11. 35-40). "Thus, it is possible to record 
the monitoring image data having a high degree of importance as much as 
possible" (id., col. 7, 11. 40-42) (emphasis added) . 

Therefore, Metcalfe in view of Takahashi and Matsumoto does not 
store an image signal in place of a stored image when the value of the 
saliencv signal for the image signal is greater than the value of the 
saliencv signal for the stored image, in contradistinction to claim 40 . 
Rather, Metcalfe in view of Takahashi and Matsumoto compresses image 
signals having lower saliency signals more than it compresses image 
signals having higher saliency signals. However, when the memory is full 
in Metcalfe in view of Takahashi and Matsumoto, a more salient image 
signal does not replace a less salient image signal. Indeed, Metcalfe in 
view of Takahashi and Matsumoto makes this explicitly clear -while stored 
data is recompressed depending on its importance to increase residual 
memory capacity, image data having a high degree of importance is 
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recorded only as much as possible. If there is no residual memory 
capacity remaining, in other words, existing, less salient image data is not 
replaced by more salient image data, in contradistinction to claim 40 . 

Therefore, Metcalfe in view of Takahashi and Matsumoto does not 
suggest the subject matter of claim 40, such that Metcalfe in view of 
Takahashi and Matsumoto does not render claim 1 prima facie obvious 
and unpatentable. 

> The Examiner disagrees. As shown in col. 4, lines 1 1-19, 
Matsumoto et al. teach "Moreover, when the data storage means has no 
residual capacity and an image having a high degree of monitoring 
importance is fetched, image data having a low degree of monitoring 
importance which are stored in the data storage means are identified 
through a flag attached every image data, for example, and are deleted or 
overwritten. Thus, image data having a high degree of monitoring 
importance are stored in the data storage means ." that based on the 
degree of importance images can be replaced with other images. 
Therefore, Matsumoto et al. teach that the memory stores the image 
signal in place of a stored image when the value of the saliency signal is 
greater than a value of a second saliency signal associated with the stored 
image and the memory is full as claimed. 
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Claim Objections 

5. The numbering of claims is not in accordance with 37 CFR 1 .1 26 which 
requires the original numbering of the claims to be preserved throughout the 
prosecution. Claim 60 does not appear on the list of claims. It is noted that claim 
60 was presented on the response filed on December 17, 2007 and was 
cancelled on the communication of January 17, 2008. 

Claim Rejections - 35 USC §112 

6. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and 
process of making and using it, in such full, clear, concise, and exact terms as to enable any 
person skilled in the art to which it pertains, or with which it is most nearly connected, to make 
and use the same and shall set forth the best mode contemplated by the inventor of carrying 
out his invention. 

7. Claims 1-5, 7, 11-20, 44, 46 and 47 are rejected under 35 U.S.C. 112, 
first paragraph, as failing to comply with the enablement requirement. The 
claim(s) contains subject matter which was not described in the specification in 
such a way as to enable one skilled in the art to which it pertains, or with which it 
is most nearly connected, to make and/or use the invention. 

8. Regarding claim 1 , it is noted that the limitations "wherein the at least 
one operation comprises automatically activating the electronic camera, without 
user interaction, to produce the image signal the image signal representing a still 
digital photograph " are considered negative limitations in which the operation is 
an operation that is not made by the user. It is also noted that any negative 
limitation or exclusionary proviso must have basis in the original disclosure. If 
alternative elements are positively recited in the specification, they may be 
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explicitly excluded in the claims. See In re Johnson, 558 F.2d 1008, 1019, 194 
USPQ 187, 196 (CCPA 1977) ("[the] specification, having described the whole, 
necessarily described the part remaining."). See also Ex parte Grasselli, 231 
USPQ 393 (Bd. App. 1983), aff 'd mem., 738 F.2d 453 (Fed. Cir. 1984). The 
mere absence of a positive recitation is not basis for an exclusion . Any claim 
containing a negative limitation which does not have basis in the original 
disclosure should be rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. Note that a lack of literal basis in 
the specification for a negative limitation may not be sufficient to establish a 
prima facie case for lack of descriptive support. Ex parte Parks, 30 USPQ2d 
1234, 1236 (Bd. Pat. App. & Inter. 1993). See MPEP 2173.05(i). 

9. Regarding claims 2-5, 7 and 11-20, claims 2-5, 7 and 11-20 are rejected 
under 35 U.S.C. 112, first paragraph as they depend from claim 1 which is 
rejected under 35 U.S.C. 112, first paragraph. 

1 0. Regarding claim 44, claim 44 has the same issues as claim 1 . 

1 1 . Regarding claims 46 and 47, claims 46 and 47 are rejected under 35 
U.S.C. 112, first paragraph as they depend from claim 44 which is rejected under 
35 U.S.C. 112, first paragraph. 
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12. The following is a quotation of the second paragraph of 35 U.S. C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

13. Claims 1-5, 7, 11-20, 44, 46 and 47 are rejected under 35 U.S.C. 112, 
second paragraph, as being indefinite for failing to particularly point out and 
distinctly claim the subject matter which applicant regards as the invention. 

14. Regarding claim 1, claim 1 recites the following: 

An apparatus comprising: 

an electronic camera to produce an image signal; 

a first user operable control to selectively activate the 
electronic camera to take pictures; 

a second user operable control to receive an input from a 
user and to generate, in response to the input from the user, a 
saliencv to indicate user interest in a picture, the saliency signal to 
have at least one of (a) a value selected from at least three different 
discrete; or (b) a value selected from a continuous range of values; 

a circuit to record the value of the saliency signal based on 
the input received via the second control contemporaneously with 
activation of the first control; and 

a memory arranged to store the image signal and the 
saliency signal, 

wherein at least one operation of the apparatus is controlled 
based on the saliencv signal, the at least one operation being 
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different from recording the value of the saliencv signal in the 
memory, and 

wherein the at least one operation comprises automatically 
activating the electronic camera, without user interaction, to 
produce the image signal, the image signal representing a still 
digital photograph . 

The Examiner noted that the claim appears to indicate that the operation 
performed by the camera is performed based on a saliency signal that is inputted 
by the user and also indicates that the operation is for automatically capturing an 
image signal representing a still image without user interaction rendering the 
claim indefinite as it contradicts itself since the capture of the still image is based 
on the saliency signal that is user operated. 

15. Regarding claims 2-5, 7 and 11-20, claims 2-5, 7 and 11-20 are rejected 
under 35 U.S.C. 1 12, second paragraph as they depend from claim 1 which is 
rejected under 35 U.S.C. 112, second paragraph. 

16. Regarding claim 44, claim 44 has the same issues as claim 1 in which 
the claim appears to indicate that the operation performed by the camera is 
performed based on a saliency signal that is inputted by the user and also 
indicates that the operation is for automatically capturing an image signal 
representing a still image without user interaction rendering the claim indefinite 
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as it contradicts itself since the capture of the still image is based on the saliency 
signal that is user operated. 



17. Regarding claims 46 and 47, claims 46 and 47 are rejected under 35 
U.S.C. 112, first paragraph as they depend from claim 44 which is rejected under 
35 U.S.C. 112, first paragraph. 



Claim Rejections - 35 USC § 103 

18. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

19. Claims 44, 46 and 47 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Takahashi, US 2002/0041757 A1 in view of Mann, US 
Patent 6,614,408 B1. 

20. Regarding claim 44, Takahashi discloses an imaging system comprising 
an electronic camera (See figs. 4(a), 4(b), 8, 9(a), 9(b), 10, and 11) to produce an 
image signal representative of a viewed scene, a physically or mechanically 
operable user control (Takahashi discloses the use of buttons 109 to set auxiliary 
information (which the Examiner is interpreting as the saliency information) 
related to the image data being captured; page 6, U 0106. Takahashi further 
discloses the use of a pressure sensor 109a and sweat sensor 109b to 
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determine the auxiliary information related to the image data (page 6,^0112- 
page 7, U 0114) to receive an input from a user and generate a first saliency 
signal while the image signal is being produced (Takahashi further discloses that 
the auxiliary information includes information related to the persons (i.e. son, 
daughter, friend, father, mother) (see figs. 23(a) and 23 (b); page 8,^0117- 
page 9, U 0124)) to receive an input from a user and to generate, in response to 
the input from the user a saliency signal (auxiliary information having information 
such as persons information)), saliency circuitry to automatically generate an 
image related second saliency signal in response to the image signal {Takahashi 
further teaches that the auxiliary information includes a degree of importance of 
said persons appearing in the image data, and that said degree of importance 
can be determined based on the time length of a scene where a particular person 
set by the user appear; see page 8, 01 19. Takahashi further teaches that the 
level of importance of a particular scene would have a value between 0(0 = not 
important) and 1 (1 = most important) with increments of 0. 1 that can be 
determined based on the excitement of the user by using a pressure sensor of a 
sweat sensor or by having a measurement of loudness in the scene being 
captured (See U 01 19). Takahashi further discloses that the level of importance 
which can be set manually could also be set automatically by use of said 
pressure, sweat or loudness sensors (Also in ]| 0119)), and circuitry to combine 
said first and second saliency signals while the image signal is being produced to 
provide a composite saliency signal (Takahashi discloses that the auxiliary 
information has persons information and degree of importance of said persons 



Application/Control Number: 1 0/696,567 Page 1 1 

Art Unit: 2622 

and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary information on a 
display screen (As shown in figs. 23, the user would set the auxiliary information 
and the set auxiliary information would be displayed in the display as shown in 
figs. 25 and 26 (see persons information and the degree of importance displayed 
on the display); page 8, ^0118- page 9, U 0124)) (This teaches the combination 
of the two saliency signals while the image signal is being produced (the persons 
information and the degree of importance information stored/displayed together 
in association with the image while the image is being captured upon operation of 
the user) to create a composite saliency signal as claimed. Therefore, by 
teaching that the person's information and the degree of importance information 
is stored/displayed together in association with the image, Takahashi discloses 
circuitry for combining said first and second saliency signals while the image 
signal is being produced to provide a composite saliency signal as claimed), 
wherein the first saliency signal, the second saliency signal, and the composite 
saliency signal are to indicate an amount of user interest in the viewed scene 
(Considering that the user sets the first saliency signal and the second saliency 
signal {As discussed above, information related to the persons (i.e. son, 
daughter, friend, father, mother) and the level of importance of a particular scene 
would have a value between 0(0 = not important) and 1 (1 = most important) 
with increments of 0. 1 that can be determined based on the excitement of the 
user by using a pressure sensor of a sweat sensor or by having a measurement 
of loudness in the scene being captured (See U 01 19) and as shown in figs. 25 
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and 26, the composite saliency signal that is displayed on the screen having the 
first and second saliency signals combined to illustrate the level of importance of 
the scene being produced (page 8, ^0118- page 9, U 0124)), and the composite 
saliency signal is to be used to control operation of at least a part of the imaging 
system (Takahashi discloses that the auxiliary information has persons 
information and degree of importance of said persons and that the auxiliary 
information is stored with the image signal and further discloses that the camera 
would also display the auxiliary information on a display screen (As shown in figs. 
23, the user would set the auxiliary information and the set auxiliary information 
would be displayed in the display as shown in figs. 25 and 26 (see persons 
information and the degree of importance displayed on the display); page 8, H 
0118 - page 9, U 0124)) being arranged to be controlled in response to the 
composite saliency signal (based on the auxiliary information the display would 
display the auxiliary information as shown in figs. 25 and 26). 

Takahashi does not explicitly disclose that the at least one operation 
comprises automatically activating the electronic camera, without user 
interaction, to produce the image signal, the image signal representing a still 
digital photograph. 

However, Mann discloses an imaging device {Fig. 2a) that captures video 
that is being stored in a circular buffer and that measure the user's pupil to 
determine a particular level of interest on the image being captured so that when 
the pupil is dilated more than normal for a particular light level, image capture is 
performed and stored or transmitted (Col. 12, lines 12-64). Mann further teaches 
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that the invention would allow a better photographic composition because the act 
of taking pictures or shooting video no longer requires conscious thought or effort 
(Co/. 21, lines 20-46). 

Therefore, taking the combined teaching of Takahashi in view of Mann as 
a whole, it would have been obvious to one of an ordinary skill in the art at the 
time the invention was made to apply the concept of having a camera that 
capture images based on a level of importance determined measured from the 
user's pupil dilatation without the user physically controlling the capture of 
images as taught by Mann to modify the teaching of Takahashi et al. to 
automatically activating the electronic camera, without user interaction, to 
produce the image signal, the image signal representing a still digital photograph. 
The motivation to do so would have been to allow a better photographic 
composition because the act of taking pictures or shooting video no longer 
requires conscious thought or effort as suggested by Mann (Col. 21, lines 20-46). 

21 . Regarding claim 46, Takahashi discloses a user operable picture taking 
control to enable the electronic camera to take pictures {shooting button 104 as 
shown in fig. 4 (b)). 

22. Regarding claim 47, Takahashi discloses that the first saliency signal is 
to include more than two values (As shown in figs. 23 (a) and 23 (b), Takahashi 
discloses that more than one person can be selected (i.e. the son and the 
daughter); page 8, H 071 9). 
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23. Claims 1, 3-5 and 11-20 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Metcalfe, AU 743216 B in view of Takahashi, US 
2002/0041757 A1 and further in view of Mann, US Patent 6,614,408 B1. 

24. Regarding claim 1, Metcalfe discloses a camera apparatus (See fig. 1) 
comprising an electronic camera (See fig. 1) to produce an image signal, a first 
user operable control (111 as shown in fig. 1) to selectively activate the electronic 
camera to take pictures, and a second user operable control (112 as shown in 
fig. 1, note that the limitations "an additional physically or mechanically operable 
user control" are written as optional elements by using the word "or') to receive 
an input from a user and to generate, in response to the input from the user a 
saliency signal to indicate user interest in a picture (Metcalfe discloses 
generating a level of interest (LOI) set by the user when using button 112), the 
saliency signal to have at least one of (a) a value selected from at least three 
different discrete values (Metcalfe discloses assigning a level of interest to the 
image data being recorded, wherein the user can variably assign a plurality of 
level of interest through the capture of the video sequence; see Fig. 3. This 
teaches generating a saliency signal that is selected from at least three different 
discrete values while the image signal is being produced), or (b) a value selected 
fro a continuous range of values (Metcalfe discloses assigning a level of interest 
to the image data being recorded, wherein the user can variably assign a plurality 
of level of interest through the capture of the a video sequence; see Fig. 3. This 
teaches generating a saliency signal that have a value selected fro a continuous 
range of values), a circuit to record the value of the saliency signal based on the 
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input received via the second control contemporaneously with activation of the 
first control (Metcalfe discloses that the LOI is stored as the frames are captured 
by the video camera 101 (Page 5, lines 8-35) and also that after operation of the 
record button 111, the LOI signals can be input by the user and recorded by the 
video camera (Page 4, lines 5-32). This inherently teaches a circuitry to record 
the value of the saliency signal based on the input received via the second 
control contemporaneously with activation of the first control as claimed since a 
circuitry is necessary to perform the recording operation of the LOI to the tape 
and also considering that the operation to record LOI signals can be performed 
right after the record button to capture image frames is operated as taught in 
Metcalfe. It is also noted that the word "contemporaneous" is defined as 
"Originating, existing, or happening during the same period of time" (Definition 
from "The American Heritage® Dictionary of the English Language", Fourth 
Edition, 2000)), and a memory arranged to store the image signal and the 
saliency signal (video cassette120 as shown in fig. 1; page 4, lines 5-13; page 5, 
lines 4-35), wherein at least one operation of the apparatus is controlled based 
on the saliency signal (Metcalfe discloses setting the LOI while the camera is 
activated to take pictures, and storing said LOI associated with the images so 
that said LOI can be used to control the reproduction of the images (i.e. printing, 
creating thumbnail files for photo albums, etc.)) being arranged to be controlled in 
response to the saliency signal (as discussed in page 6, line 31 - page 7, line 8 , 
Metcalfe discloses that the saliency signal (LOI) can be used to control the 
reproduction of the images (i.e. printing, creating thumbnail files for photo 
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albums, etc.)) (See page 4, line 5 - page 5, line 23; page 6, line 23 - page 7, line 
8). 

Metcalfe does not explicitly disclose that the at least one operation is 
different from recording the value of the saliency signal in the memory and that 
that the at least one operation comprises automatically activating the electronic 
camera, without user interaction, to produce the image signal, the image signal 
representing a still digital photograph. 

However, Takahashi discloses an electronic camera {See figs. 4(a), 4(b), 
8, 9(a), 9(b), 10, and 11) producing image signal, comprising a first user operable 
control (shooting button 104 as shown in fig. 4 (b)) to selectively activate the 
electronic camera to take pictures, and a second user operable control 
(Takahashi discloses the use of buttons 109 to set auxiliary information (which 
the Examiner is interpreting as the saliency information) related to the image data 
being captured; page 6, ^0106. Takahashi further discloses the use of a 
pressure sensor 109a and sweat sensor 109b to determine the auxiliary 
information related to the image data (page 6, ^0112- page 7, U 01 14). Also 
Takahashi discloses that the auxiliary information includes information related to 
the persons (i.e. son, daughter, friend, father, mother) and a degree of 
importance of said persons appearing in the image data (see figs. 23(a) and 23 
(b); page 8,^0117- page 9, U 0124)) to receive an input from a user and to 
generate, in response to the input from the user a saliency signal to indicate user 
interest in a picture (auxiliary information having information such as persons 
information and degree of importance of the persons in the image), the saliency 
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signal to have at least one of (a) a value selected from at least three different 
discrete values (as shown in figs. 23(a) and 23(b), the persons information and 
degree of importance of the persons in the image can change in value between a 
plurality of values (i.e. three or more values as claimed)), or (b) a value selected 
from a continuous range of values (the auxiliary information can be changed 
during the capture of the image signal to a plurality of continuous values as 
shown in figs. 23(a) and 23(b)), and a memory (Fig. 1: 13) arranged to store the 
image signal and the saliency signal (Takahashi further teaches storing the 
auxiliary information in the header of the scene; page 8, U 0123 - page 9, U 
0126), wherein at least one operation of the apparatus is controlled based on the 
saliency signal (Takahashi discloses recording the auxiliary information and 
further discloses that the camera would also display the auxiliary information on a 
display screen (As shown in figs. 23, the user would set the auxiliary information 
and the set auxiliary information would be displayed in the display as shown in 
figs. 25 and 26; page 8, ^0118- page 9, U 0124). It is noted that by teaching 
that based on the auxiliary information the display would display the auxiliary 
information Takahashi discloses that at least one operation of the apparatus is 
controlled based on the saliency signa\), the at least one operation being different 
from to recording the saliency signal in the memory (The Examiner is interpreting 
displaying the auxiliary information using the display of the camera as the 
operation that is different from recording the saliency signal in the memory as 
claimed). Displaying the auxiliary information while capturing the image data is 
advantageous because it would provide the user with a user friendly interface 
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that would allow changing the degree of importance of the images being 
captured and would also allow the user to be aware of the information being 
added to the image data. 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi 
as a whole, it would have been obvious to one of an ordinary skill in the art at the 
time the invention was made to apply the concept of having the camera further 
displaying the salient signal being assign to the image data as taught in 
Takahashi to modify the teaching of Metcalfe to perform an operation in response 
to the saliency signal, the operation being different from recording the saliency 
signal in the memory. The motivation to do so would have been to provide the 
user with a user friendly interface that would allow changing the degree of 
importance of the images being captured and would also allow the user to be 
aware of the information being added to the image data. 

The combined teaching of Metcalfe in view of Takahashi fails to teach that 
the at least one operation comprises automatically activating the electronic 
camera, without user interaction, to produce the image signal, the image signal 
representing a still digital photograph. 

However, Mann discloses an imaging device {Fig. 2a) that captures video 
that is being stored in a circular buffer and that measure the user's pupil to 
determine a particular level of interest on the image being captured so that when 
the pupil is dilated more than normal for a particular light level, image capture is 
performed and stored or transmitted (Col. 12, lines 12-64). Mann further teaches 
that the invention would allow a better photographic composition because the act 
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of taking pictures or shooting video no longer requires conscious thought or effort 
(Co/. 21, lines 20-46). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi 
and further in view of Mann as a whole, it would have been obvious to one of an 
ordinary skill in the art at the time the invention was made to apply the concept of 
having a camera that capture images based on a level of importance determined 
measured from the user's pupil dilatation without the user physically controlling 
the capture of images as taught by Mann to modify the teaching of Metcalfe and 
Takahashi et al. to automatically activating the electronic camera, without user 
interaction, to produce the image signal, the image signal representing a still 
digital photograph. The motivation to do so would have been to allow a better 
photographic composition because the act of taking pictures or shooting video no 
longer requires conscious thought or effort as suggested by Mann (Col. 21, lines 
20-46). 



25. Regarding claim 3, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann as discussed and analyzed in claim 1 
further teaches a buffer to receive the image signal, the buffer having a capacity 
controlled by the value of the saliency signal {In a further embodiment, Takahashi 
discloses the concept of determining the amount of data to be transmitted based 
on the cost of transmission service; wherein the allowed length of data (L) is 
determined and compared to the amount of data to be transmitted, if the amount 
of data to be transmitted the apparatus would select video section with a priority 
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higher than a threshold value. If after selecting those video with the priority 
higher than a threshold, the length of data still more than the allowed length, the 
apparatus would increase the priority threshold, however, if the length is less that 
the allowed length, the video signals are transmitted (Page 12, U 0151-0154). By 
teaching adjusting the amount of data to be transmitted based on the length 
allowed for transmission and also based on the priority of the video signals to be 
transmitted, Takahashi inherently discloses "a buffer to receive the image signal, 
the buffer having a capacity controlled by the value of the saliency signal" as 
claimed since the amount being adjusted in the apparatus needs to be in a 
particular buffer or memory prior to transmission in order to properly select the 
video signals with higher priority). One of an ordinary skill in the art would have 
found obvious to apply the concepts of adjusting the amount of data to be 
transmitted based on the priority set to the video signals to further modify the 
camera apparatus to have a buffer for receiving said image signal, the buffer 
having a capacity arranged to be controlled by the value of the saliency signal 
during operation of the camera apparatus with the motivation of select as many 
as possible the most important video signals from the video data for transmission 
as suggested by Takahashi {Page 12, ^0154). 

26. Regarding claim 4, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann as discussed and analyzed in claim 1 
further teaches image selection circuitry to receive the saliency and image 
signals and to selectively pass the image signal based on the saliency signal (As 
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taught in Metcalfe, the camera receives the saliency signal (LOI) and based on 
said saliency signal, when reproducing, the camera would select particular 
images based on the degree of importance as set by the user when recording the 
images. By teaching that the images with a high LOI would be selected for 
creating an album or to be printed with high quality further teaches that images 
that do not have a high LOI would be passed or skipped since only the images 
with a high LOI would be displayed in the virtual album. Metcalfe as applied 
reads on ". . .image selection circuitry to receive the saliency and image signals 
and to selectively pass the image signal based on the saliency signal" as claimed 
since the display or skip of images is determined based on the LOI set. (See 
pages 5-7, specifically page 6, line 31 -page 7, line 8)). 

27. Regarding claim 5, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann as discussed and analyzed in claim 1 
further teaches management circuitry to selectively retain in the memory 
(memory 13 in Takahashi, fig. 1) images associated with higher saliency levels in 
preference to images with lower saliency levels (As discussed with respect to 
claim 1, Takahashi further discloses that the images are recorded or transmitted 
based on the importance level of the image, wherein only images with high 
importance level can be recorded in order to reduce the amount of use of the 
recording medium; page 5, U 0094). It would have been obvious to one of an 
ordinary skill in the art at the time the invention was made to modify the teaching 
of Metcalfe with the concepts and teaching of Takahashi to further include 
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management circuitry to selectively retain in the memory, images associated with 
higher saliency levels in preference to images with lower saliency levels. The 
motivation to do so would have been to reduce the amount of use of the 
recording medium as suggested by Takahashi {Page 5, U 0094). 

28. Regarding claim 1 1 , Metcalfe discloses that the first user control includes 
a normal picture taking control on the electronic camera {111 as shown in fig. 1). 

29. Regarding claim 12, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann as discussed and analyzed in claim 1 
further teaches that the saliency signal is a first saliency signal and further 
comprising a third user operable control to generate a second saliency signal {As 
shown in Takahashi, figs. 4(b): 109; fig. 9(a): 109, 10: 109a, and 11: 109b, 
Takahashi discloses the use of a plurality of buttons to select from different 
auxiliary information to be assigned to the video signal (note that the buttons 109 
are physically located on the camera). Furthermore, as shown in figs 23(a) and 
23(b), Takahashi further discloses that the auxiliary information can be selected 
using a monitor, wherein the user can select the person information and the 
degree of importance of the persons using buttons 103m, 103n, 101m and 101n; 
page 8, U 0119 (although these buttons are displayed on the touch screen 
display, the Examiner understands that the buttons are considered different user 
controls since they have different physical location on said display)). Grounds for 
rejecting claim 1 apply here. 
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30. Regarding claim 13, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann as discussed and analyzed in claims 1 
and 1 2 further teaches saliency circuitry to combine the first and second signals 
to form a complex saliency signal (the Examiner is reading the complex saliency 
signal as the combined information having the persons information and the 
degree of importance information as shown in Takahashi), the complex saliency 
signal being the saliency signal to control at least one operation [The Examiner is 
reading the operation as the display operation in the camera in Takahashi as 
discussed in claim 1) and the saliency signal stored by the memory {As 
discussed in claim 1, Takahashi discloses that the auxiliary information has 
persons information and degree of importance of said persons and that the 
auxiliary information is stored with the image signal and further discloses that the 
camera would also display the auxiliary information on a display screen (As 
shown in figs. 23, the user would set the auxiliary information and the set 
auxiliary information would be displayed in the display as shown in figs. 25 and 
26 (see persons information and the degree of importance displayed on the 
display); page 8, y[0118- page 9, U 0124)) being arranged to be controlled in 
response to the saliency signal (based on the auxiliary information the display 
would display the auxiliary information as shown in figs. 25 and 26. Furthermore, 
the plurality of saliency signals are both stored in the memory and the display 
operation of the camera is performed based on the first and second saliency 
signals as they are input and stored in memory). Grounds for rejecting claim 1 
and 12 apply here. 
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31 . Regarding claim 14, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann as discussed and analyzed in claim 1 
further teaches that the saliency signal is a first saliency signal (As discussed in 
claim 1, the saliency signal in Metcalfe for controlling the level of interest of a 
captured scene is considered a first saliency signal as claimed. Furthermore, 
Takahashi further teaches the sue of a plurality of saliency signals to be used to 
control the operation of the camera as discussed in claim 1) and further 
comprising a saliency circuitry to generate an image related saliency signal in 
response to said image signal (Metcalfe discloses controlling the reproduction 
operation of the camera based on the associated saliency signal to the image 
signal so that when reproducing the image signal with higher importance would 
be displayed differently from the other image signal. See page 4, line 5 - page 5, 
line 23; page 6, line 23 - page 7, line 8. Takahashi further discloses reproducing 
the video based on the importance level of the image, wherein only images with 
high importance level can be reproduced so that the user can enjoy the recorded 
work without feeling tired, and the power consumption is reduced to secure more 
driving time; page 10, U 01 31). 

32. Regarding claim 15, limitations have been discussed and analyzed in 
claim 13. 

33. Regarding claim 16, Metcalfe discloses circuitry to incorporate the 
saliency signal in the image signal (Metcalfe discloses incorporating said saliency 
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signal (LOI) to each of the frames in the image signal. See page 4, line 5 - page 
5, line 23; page 6, line 23 - page 7, line 8). 



34. Regarding claim 17, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann as discussed and analyzed in claims 1 
teaches that the second user control is part of the camera or is physically 
attached to the camera body (See Metcalfe, controls in Fig. 1; see also 
Takahashi buttons 109 as shown in figs. 4(b), 9(b), 10 and 11, and buttons 
103m, 103n, 101m, 101 n as shown in figs. 23(a) and 23(b)). 



35. Regarding claim 18, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Mann fails to teach that the second user control 
is a remote control for communication with the camera. However, the Examiner 
takes Official Notice that the concept of controlling a camera with a remote 
control is well known in the art at the time the invention was made and that it 
would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have made the user control of Metcalfe and Takahashi a 
remote control as opposed to a camera-body integrated control. One would have 
been motivated to do so because it is well known in the art that by using a 
remote control to control some elements of a camera, the user does not have to 
be near the camera to send and receive desired signals from the camera. This is 
particularly advantageous in cases where plural cameras are used or cameras 
are placed out of the reach of the user (e.g. surveillance cameras), where the 
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remote control would allow the user to send signals to the camera(s) from a 
separate location, thereby simplifying camera control for the user. It is noted that 
in the previous Office Action, claim 18 was rejected taking Official Notice to the 
recited limitations. Because the Applicant failed to traverse the Examiner's 
assertion of Official Notice, the well known in the art statement is taken to be 
admitted prior art. See MPEP § 2144.03 [R-1] (C). 

36. Regarding claim 19, Metcalfe discloses that the second user control 
comprises a physically movable control member (pressure button that assign a 
level of interest of an image signal based on the pressure applied to said button) 
and a sensor arranged to be responsive to movement of the control member 
(See page 4, line 27 -page 5, line 4). By teaching a pressure button that 
assigns a LOI of an image signal based on the pressure applied to the button, 
Metcalfe discloses a physically movable control member (in this case the 
pressure button as discussed in Metcalfe) and a sensor arranged to be 
responsive to movement of the control member since by teaching that the LOI is 
assigned based on the pressure applied to the button, the use of a sensor to 
determine the applied pressure in order to assigned the LOI of an image is 
inherent and necessitated in Metcalfe). 

37. Regarding claim 20, the Examiner notes that the limitations "the second 
user control comprises a pressure or force sensing transducer for deriving the 
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saliency signal that can have values that are continuously variable", the elements 
"pressure" or "force sensing transducer" are written as optional elements by 
using the word "or". 

Metcalfe discloses that the second user control comprises a pressure or 
force sensing transducer for deriving the saliency signal that can have values 
that are continuously variable (Metcalfe as applied to claim 19, teaches the use 
of pressure (pressure button that assign a level of interest of an image signal 
based on the pressure applied to said button; page 4, line 27 - page 5, line 4) to 
determine the value of the saliency signal. By teaching that Metcalfe discloses 
the use of pressure applied to the button, Metcalfe discloses the use of pressure 
to derive the value of the saliency signal as claimed. 

38. Claims 2 and 7 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Metcalfe, AU 743216 B and Takahashi, US 2002/0041757 
A1 in view of Mann, US Patent 6,614,408 B1 and further in view of 
Matsumoto et al., US Patent 6,795,642 B2. 

39. Regarding claim 2, although Takahashi teaches a compression circuitry 
[See Takahashi, fig. 1: 15; page 5, U 0097; page 6, U 0105) to compress the 
image signals and that although the invention is described on the premise that a 
shot picture is recorded, a shot picture is not necessarily recorded, and it can be 
used also when compressed video and audio data are transmitted as they are to 
be used on a network or the like {page 10, U 0130), the combined teaching of 
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Metcalfe in view of Takahashi and further in view of Mann fails to teach 
compressing the image signals to an extent determined by the saliency signal. 

However, Matsumoto et al. teaches the concept of having a video 
recording apparatus (Fig. 2) of a surveillance system, recording video data 
captured by an electronic camera (See fig. 1), wherein when an alarm is 
activated, the importance of the video is determined to be high as compared to 
when the alarm is not activated (Col. 3, lines 46-67). Matsumoto et al. further 
discloses that based on the degree of importance given to the video signal, the 
data compression is also adjusted (i.e. if the importance degree of the video is 
low, it would be compressed at high level and if the importance degree of the 
video is high, said video would be compressed at low level) (Col. 3, line 15- col. 
4, lines 19). Matsumoto also discloses that the importance level can also be 
adjusted by the user operating the surveillance system (Col. 7, lines 22-34). 
Matsumoto et al. further discloses that by adjusting the compression of the video 
being captured, it is possible to record the monitoring image data having a high 
degree of importance as much as possible (Col. 7, lines 35-43). 

Therefore, taking the combined teaching of Metcalfe and Takahashi in 
view of Mann and further in view of Matsumoto et al. as a whole, it would have 
been obvious to one of ordinary skill in the art a the time the invention was made 
to apply the concept of adjusting the compression of a video captured by an 
electronic camera based on a degree of importance assign to the video as 
discussed in Matsumoto et al. to modify the teaching of Metcalfe, Takahashi and 
Mann by compressing the image signals to an extent determined by the saliency 
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signal. The motivation to do so would have been to record the monitoring image 
data having a high degree of importance as much as possible as suggested in 
Matsumoto et al. (Col. 7, lines 35-43). 

40. Regarding claim 7, the combined teaching of Metcalfe and Takahashi in 
view of Mann and further in view of Matsumoto et al. further teaches a 
management circuitry to selectively retaining in the memory images associated 
with higher saliency levels in preference to images with lower saliency levels 
{Takahashi further discloses that the images are recorded or transmitted based 
on the importance level of the image, wherein only images with high importance 
level can be recorded in order to reduce the amount of use of the recording 
medium; page 5, U 0094). 

41. Claims 40 and 42 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Metcalfe, AU 743216 B in view of Takahashi, US 
2002/0041757 A1 and further in view of Matsumoto et al., US Patent 
6,795,642 B2. 

42. Regarding claim 40, Metcalfe discloses an imaging system (See fig. 1) 
comprising an electronic camera (See fig. 1) to produce an image signal, 
physically or mechanically operable user controls (See user controls 106, 104, 
111, 112, 110 and 108 as shown in fig. 1), the user controls being arranged to 
receive an input from a user and to generate first saliency signal (Metcalfe 
discloses the use of button 112 to generate a plurality of saliency signals (Level 
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of interest signals "LOI") to be associated to the image signal being recorded with 
the camera) while the image signal is being produced, and saliency circuitry (the 
camera in Metcalfe inherently has a saliency signal circuitry to generate the 
saliency signal upon operation of the camera button 112) for storing said first 
saliency signal (Metcalfe discloses recording the plurality of saliency signals in a 
memory (tape 120 in fig. 1) in association with the image data; see page 4, line 5 
- page 5, line 23; page 6, line 23 - page 7, line 8), the saliency signal having at 
least one of (a) a value selected from at least three different discrete values 
(Metcalfe discloses assigning a level of interest to the image data being 
recorded, wherein the user can variably assign a plurality of level of interest 
through the capture of the a video sequence; see Fig. 3. This teaches 
generating a saliency signal having a value from at least three different discrete 
values), or (b) a value selected from a continuous range of values (Metcalfe 
discloses assigning a level of interest to the image data being recorded, wherein 
the user can variably assign a plurality of level of interest through the capture of 
the a video sequence; see Fig. 3. This teaches generating a saliency signal 
having a value selected from a continuous range of values), a memory (120 as 
shown in fig. 1) to store the image signal and the saliency signal in response to 
the saliency signal (page 4, lines 5-13; page 5, lines 4-35), and operation of at 
least part of the electronic camera being arranged to be controlled in response to 
the saliency signal (as discussed in page 6, line 31 - page 7, line 8, Metcalfe 
discloses that the saliency signal (LOI signal stored in memory 120) can be used 
to control the reproduction of the images (i.e. printing, creating thumbnail files for 
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photo albums, etc.)) (See page 4, line 5 - page 5, line 23; page 6, line 23 - page 
7, line 8). 

Metcalfe does not explicitly disclose at least two physically or 
mechanically operable user controls, each of the user controls for generating first 
and second saliency signals; that said saliency circuitry combines said first and 
second saliency signals to form a complex saliency signal; and that said memory 
stores the image signal in place of a stored image when the value of the saliency 
signal is greater than a value of a second saliency signal associated with the 
stored image and the memory is full. 

However, Takahashi teaches an imaging system comprising an electronic 
camera (See figs. 4(a), 4(b), 8, 9(a), 9(b), 10, and 11) to produce an image signal 
representative of a viewed scene, a physically or mechanically operable user 
control (Takahashi discloses the use of buttons 109 to set auxiliary information 
(which the Examiner is interpreting as the saliency information) related to the 
image data being captured; page 6, U 0106. Takahashi further discloses the use 
of a pressure sensor 109a and sweat sensor 109b to determine the auxiliary 
information related to the image data (page 6, ^0112- page 7, U 0114) to 
receive an input from a user and generate a first saliency signal while the image 
signal is being produced (Takahashi further discloses that the auxiliary 
information includes information related to the persons (i.e. son, daughter, friend, 
father, mother) (see figs. 23(a) and 23 (b); page 8,^0117 - page 9, H 0124)) to 
receive an input from a user and to generate, in response to the input from the 
user a saliency signal (auxiliary information having information such as persons 
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information)), saliency circuitry to automatically generate an image related 
second saliency signal in response to the image signal (Takahashi further 
teaches that the auxiliary information includes a degree of importance of said 
persons appearing in the image data, and that said degree of importance can be 
determined based on the time length of a scene where a particular person set by 
the user appear; see page 8, ^0119. Takahashi further teaches that the level of 
importance of a particular scene would have a value between 0(0 = not 
important) and 1 (1 = most important) with increments of 0. 1 that can be 
determined based on the excitement of the user by using a pressure sensor of a 
sweat sensor or by having a measurement of loudness in the scene being 
captured (See ^0119). Takahashi further discloses that the level of importance 
which can be set manually could also be set automatically by use of said 
pressure, sweat or loudness sensors (Also in ]J 0119)), and circuitry to combine 
said first and second saliency signals while the image signal is being produced to 
provide a composite saliency signal (Takahashi discloses that the auxiliary 
information has persons information and degree of importance of said persons 
and that the auxiliary information is stored with the image signal and further 
discloses that the camera would also display the auxiliary information on a 
display screen (As shown in figs. 23, the user would set the auxiliary information 
and the set auxiliary information would be displayed in the display as shown in 
figs. 25 and 26 (see persons information and the degree of importance displayed 
on the display); page 8,^0118- page 9, U 0124)) (This teaches the combination 
of the two saliency signals while the image signal is being produced (the persons 
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information and the degree of importance information stored/displayed together 
in association with the image while the image is being captured upon operation of 
the user) to create a composite saliency signal as claimed. Therefore, by 
teaching that the person's information and the degree of importance information 
is stored/displayed together in association with the image, Takahashi discloses 
circuitry for combining said first and second saliency signals while the image 
signal is being produced to provide a composite saliency signal as claimed), 
wherein the first saliency signal, the second saliency signal, and the composite 
saliency signal are to indicate an amount of user interest in the viewed scene 
(Considering that the user sets the first saliency signal and the second saliency 
signal (As discussed above, information related to the persons (i.e. son, 
daughter, friend, father, mother) and the level of importance of a particular scene 
would have a value between 0(0 = not important) and 1 (1 = most important) 
with increments of 0. 1 that can be determined based on the excitement of the 
user by using a pressure sensor of a sweat sensor or by having a measurement 
of loudness in the scene being captured (See U 0119) and as shown in figs. 25 
and 26, the composite saliency signal that is displayed on the screen having the 
first and second saliency signals combined to illustrate the level of importance of 
the scene being produced (page 8, ^0118- page 9, U 0124)), and the composite 
saliency signal is to be used to control operation of at least a part of the imaging 
system (Takahashi discloses that the auxiliary information has persons 
information and degree of importance of said persons and that the auxiliary 
information is stored with the image signal and further discloses that the camera 
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would also display the auxiliary information on a display screen (As shown in figs. 
23, the user would set the auxiliary information and the set auxiliary information 
would be displayed in the display as shown in figs. 25 and 26 (see persons 
information and the degree of importance displayed on the display); page 8, U 
0118- page 9,^0124)) being arranged to be controlled in response to the 
composite saliency signal (based on the auxiliary information the display would 
display the auxiliary information as shown in figs. 25 and 26). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi 
as a whole, it would have been obvious to one of an ordinary skill in the art at the 
time the invention was made to apply the concept of having the camera with a 
plurality of operation controls to generate different saliency signals to be 
combined to control the operation of the display to further displaying the 
combined saliency signal being assign to the image data as taught in Takahashi 
to modify the teaching of Metcalfe to have at least two physically or mechanically 
operable user controls, each of the user controls for generating first and second 
saliency signals; that said saliency circuitry combines said first and second 
saliency signals to form a composite saliency signal; that the operation of at least 
part of the electronic camera being arranged to be controlled in response to the 
complex saliency signal. The motivation to do so would have been to provide the 
user with a user friendly interface that would allow changing the degree of 
importance of the images being captured and would also allow the user to be 
aware of the information being added to the image data. 
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The combined teaching of Metcalfe in view of Takahashi fails to teach that 
the memory stores the image signal in place of a stored image when the value of 
the saliency signal is greater than a value of a second saliency signal associated 
with the stored image and the memory is full. 

However, Matsumoto et al. teaches the concept of having a video 
recording apparatus (Fig. 2) of a surveillance system, recording video data 
captured by an electronic camera (See fig. 1), wherein when an alarm is 
activated, the importance of the video is determined to be high as compared to 
when the alarm is not activated (Col. 3, lines 46-67). Matsumoto et al. further 
discloses that based on the degree of importance given to the video signal, the 
data compression is also adjusted (i.e. if the importance degree of the video is 
low, it would be compressed at high level and if the importance degree of the 
video is high, said video would be compressed at low level) (Col. 3, line 15- col. 
4, lines 19). Matsumoto also discloses that the importance level can also be 
adjusted by the user operating the surveillance system (Col. 7, lines 22-34). 
Matsumoto et al. further discloses that by adjusting the compression of the video 
being captured, it is possible to record the monitoring image data having a high 
degree of importance as much as possible (Col. 7, lines 35-43). In col. 4, lines 4, 
lines 1-19, Matsumoto et al. further teaches that based on the degree of 
importance of the image data being recorded, when the memory has no residual 
capacity images that have lower level of importance would be identified and 
deleted or overwritten so that images with high degree of importance would be 
stored in the storage means (This teaches that the memory stores the image 
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signal in place of a stored image when the value of the saliency signal is greater 
than a value of a second saliency signal associated with the stored image and 
the memory is full as claimed). 

Therefore, taking the combined teaching of Metcalfe in view of Takahashi 
and further in view of Matsumoto et al. as a whole, it would have been obvious to 
one of ordinary skill in the art a the time the invention was made to apply the 
concept of replacing images with lower level of importance with images captured 
with higher levels of importance when the memory has no residual capacity, 
images that have lower level of importance would be identified and deleted or 
overwritten as discussed in Matsumoto et al. to modify the teaching of Metcalfe 
and Takahashi by having the memory storing the image signal in place of a 
stored image when the value of the saliency signal is greater than a value of a 
second saliency signal associated with the stored image and the memory is full. 
The motivation to do so would have been to record the monitoring image data 
having a high degree of importance as much as possible as suggested in 
Matsumoto et al. (Co/. 7, lines 35-43). 

43. Regarding claim 42, the combined teaching of Metcalfe in view of 
Takahashi and further in view of Matsumoto et al. as discussed and analyzed in 
claim 40 further teaches a separate user operable picture taking control for 
selectively activating the electronic camera to take pictures (As shown in 
Takahashi, figs. 4(b): 109; fig. 9(a): 109, 10: 109a, and 11: 109b, Takahashi 
discloses the use of a plurality of buttons to select from different auxiliary 



Application/Control Number: 10/696,567 Page 
Art Unit: 2622 

information to be assigned to the video signal (note that the buttons 109 are 
physically located on the camera). Furthermore, as shown in figs 23(a) and 
23(b), Takahashi further discloses that the auxiliary information can be selected 
using a monitor, wherein the user can select the person information and the 
degree of importance of the persons using buttons 103m, 103n, 101m and 101n; 
page 8, U 0119 (although these buttons are displayed on the touch screen 
display, the Examiner understands that the buttons are considered different user 
controls since they have different physical location on said display)). 
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